Embodiments of the present invention relate generally to improving performance of interconnection networks.
Many different systems use interconnection networks to provide for communication between system components. Accordingly, interconnection networks allow communications between processors and memory modules. Interconnection networks may be used in diverse system types including large-scale shared-memory multiprocessors and clusters of loosely-coupled workstations. In some interconnection networks, communications may be encapsulated in network packets and delivered to destination processors or memories.
Many multiprocessor systems include numerous processors and large cache-coherent shared-memory. For example, some server machines have 32 or more processors and include an interconnection network to connect these processors to memory modules within the system. To operate in a desired fashion, these systems should provide for low latency and high bandwidth communication via their interconnection networks. Network packets transmitted via an interconnection network include various information. For example, processor cache misses, software messages to a remote memory module, another processor's cache, or another processor and consequent responses may occur in a given system. However, these packets can be delayed as a result of network congestion such as transient network congestion. The performance of parallel applications, such as database servers, depends on how rapidly and how many of these requests and responses can be processed.
Some networks may exhibit a phenomenon called a hot spot. A hot spot is an area of congestion in which communications are delayed. Hot spots may be detrimental to the performance of the network. Hot spots build up due to congestion at specific localized areas of a network. The generation of a hot spot may, over time, lead to tree saturation when the effect of the congestion propagates through different regions of the network.
Some systems employ different algorithms to provide for congestion control. Such algorithms typically fall into one of two categories, either acting to throttle a source to avoid injecting more traffic or routing packets around the congestion. However, these congestion control algorithms do not detect development of hot spots at locations remote to the hot spot itself. Accordingly, when hot spots arise, it is difficult to detect how the hot spot is developing in a node which is far away from a source node. Thus, the source node may continue to send messages which can further increase congestion at the hot spot. While eventually back pressure in the network may propagate to a source node targeting messages for the hot spot region, that back pressure may take so long to arrive at the source node that performance of the network may already by severely degraded.
A need thus exist for improved handling of hot spots.